4 results
1 The Mobile Toolbox for Monitoring Cognitive Change
- Richard Gershon, Cindy J. Nowinski, Aaron Kaat
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, pp. 779-780
-
- Article
-
- You have access Access
- Export citation
-
Objective:
To present the Mobile Toolbox (MTB), comprised of an expandable library of cognitive and other tests, including adapted versions of NIH Toolbox® measures. The MTB provides a complete research platform for app creation, study management, data collection, and data management. We will describe the MTB project and MTB research platform and demonstrate examples of assessments.
Participants and Methods:MTB is the product of an NIH-funded, multi-institutional effort involving Northwestern University, Sage Bionetworks, Penn State, University of California San Francisco, University of California San Diego, Emory University, and Washington University. The MTB assessment library is a dynamic repository built upon Sage Bionetworks mobile health platform. All MTB measures are created or adapted for a mobile interface using iOS and Android smartphones. Guided by the principles of open science, many components are open source to allow researchers and developers to integrate externally developed tests, including supplemental scales (e.g., passively collected contextual factors) assessing variables such as mood and fatigue that might influence cognitive test performance.
Results:The current MTB library includes eight core cognitive tests based on well-established neuropsychological measures: two language tasks (Spelling and Word Meaning), two executive functioning tasks (Arrow Matching and Shape-Color Sorting), an associative memory task (Faces and Names), an episodic memory task (Arranging Pictures), a working memory task (Sequences) and a processing speed task (Numbers and Symbols). Additional cognitive assessments from other popular test libraries including the International Cognitive Ability Resource (ICAR), Cognitive Neuroscience Test Reliability and Clinical Applications for Schizophrenia (CNTRACS) and Test My Brain are currently being implemented, as are non-cognitive measures from the NIH Toolbox Emotion Battery and the Patient-Reported Outcomes Measurement Information System (PROMIS). The MTB library includes measures suitable for use in research studies incorporating point-in-time and burst designs as well as ecological momentary assessment (EMA).
Conclusions:The MTB was created to address many of the scientific, practical, and technical challenges to cognitive assessment by capitalizing on advances in technology measurement and cognitive research. Initial psychometric evaluation of measures has been performed, and additional clinical validation is underway in studies with persons at risk for cognitive impairment or Alzheimer’s disease (AD), diagnosed with mild cognitive impairment (MCI) or AD, Parkinson’s disease, and HIV-associated Neurocognitive Disorders. Calculation of norms and reliable change indicators is in progress. The MTB is currently available to beta testers with public release planned for Summer, 2023. Clinical researchers will be able to use the MTB system to design smartphone-based test batteries, deploy and manage mobile data collection in their research studies, and aggregate and analyze results in the context of large-scale norming data.
2 Validity and Reliability of Mobile Toolbox Cognitive Assessments
- Cindy J Nowinski, Aaron Kaat, Jerry Slotkin, Erika La Forte, Yusuke Shono, Miriam Novack, Sarah Pila, Elizabeth Dworak, Stephanie R Young, Zahra Hosseinian, Hubert Adam, Richard Gershon
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, pp. 780-781
-
- Article
-
- You have access Access
- Export citation
-
Objective:
To present validation evidence for the first eight cognitive measures available through Mobile Toolbox (MTB). These measures use a remote self-administered platform to assess language, working memory, episodic memory, executive function, and processing speed.
Participants and Methods:We used two separate samples, recruited as part of a larger study, to validate MTB measures. Sample I, comprised of 92 English-speaking adults ages 18-85, was used to assess internal consistency and construct validity. Participants were first administered “gold standard” cognitive measures (Wechsler Memory Scale-IV Verbal Paired Associates I and II; Wechsler Adult Intelligence Scale-IV Symbol Search, Digit Span, Coding, and Letter-Number Sequencing; Delis-Kaplan Executive Function System Color-Word Interference Test, Peabody Picture Vocabulary Test, Wechsler Individual Achievement Test-4 Spelling, and the Wisconsin Card Sorting Test), after which they completed MTB (pre-loaded on a study-provided smartphone) on their own. Internal consistency was evaluated using measure-appropriate indices (split-half reliability, Cronbach’s alpha or IRT-based indices). Pearson correlation coefficients between MTB tests and measures of similar constructs were used to evaluate concurrent validity. For two tests with timing-dependent scores, Arrow Matching and Shape-Color Sorting, separate analyses were performed for iOS and Android devices. Sample II, with 1,120 English-speaking participants ages 18-90, was used to evaluate age-related change. Participants completed MTB measures remotely on their own smartphones, in a preset order, within a 14-day period. Spearman correlation coefficients, corrected for education, were calculated to evaluate relationships between age and test scores.
Results:Sample I participants were 67% female, 52% white, 99% non-Hispanic; average age=48 (SD= 17). Education was: < high school (1%); high school (55%); some college (21%); college (15%); graduate degree (8%). Internal consistency estimates ranged from 0.81 to 0.99. Pearson correlations between MTB and external measures ranged from 0.41 to 0.86 (all p < .01). Of the timed tests, only Shape-Color sorting showed significant score differences between Android and iOS devices. Sample II was 57% female, 13% Hispanic, 72% white, mean age = 45 (SD = 21). Education distribution was: < high school (2%); high school (34%); some college (34%), college (20%); graduate degree (11%). Measures of executive function (r = -0.50; r=-0.57) and processing speed (r= -0.61) showed the expected negative correlation with age (all p <0.001). Negative correlations, although weaker, were also seen on measures of working memory (r=-0.2) and episodic memory (r=-0.2, r=-0.37; p.<.001). Vocabulary performance improved with age (r=0.4; p<.001), while spelling scores remained stable (r=0.09).
Conclusions:Initial studies support the validity and reliability of the first eight MTB cognitive measures in two diverse samples. MTB tests showed satisfactory construct validity, as demonstrated by the associations between MTB and well-established tests. Furthermore, most MTB measures correlated with age in the expected directions. Executive function, processing speed and memory typically decrease with age and this decrease was reflected in MTB test performance. In contrast, spelling and vocabulary, typically preserved as we age, did not decrease in our sample. Our results support the use of MTB in cognitive aging research.
3 Mobile Toolbox: Enrollment of a Large Normative Sample Using the UCSF Brain Health Registry
- Rachel L Nosheny, Monica R Camacho, Bernard Landavazo, Aaron J Kaat, Zahra Hosseinian, Richard C Gershon, Michael W Weiner
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue s1 / November 2023
- Published online by Cambridge University Press:
- 21 December 2023, pp. 781-782
-
- Article
-
- You have access Access
- Export citation
-
Objective:
A critical need in the neuropsychology field is development and validation of efficient, scalable assessments of cognition. The Mobile Toolbox (MTB), a novel suite of mobile device-compatible, app-based cognitive assessments, was developed to address this need. The goals of this study were (1) To collect longitudinal normative data for the MTB assessments in a large, ethnoculturally and educationally diverse cohort; (2) To assess the feasibility and usability of remote assessment using MTB.
Participants and Methods:Participants were recruited from the UCSF Brain Health Registry (BHR), an online cohort (N>100,000) that collects longitudinal cognitive, functional, behavioral, and health data using online neuropsychological tests and self- and study-partner report surveys. BHR participants who opted to learning about additional research opportunities were sent automated email invitations to enroll in the MTB study. Those who indicated study interest were provided instructions within the BHR online portal for downloading the MTB app. All participants had the opportunity to complete a single baseline administration of MTB (Word Meaning, Sequences, Spelling, Arranging Pictures, Arrow Matching, Faces and Names, Shape-Color Sorting, Number Match). Those who completed the baseline assessment within three days were invited to continue into the longitudinal study, where they complete MTB assessments at a single, short-term timepoint (day 7, 14, or 21; study arms sequentially assigned), and then at 6-month intervals. Enrollment across demographic groups was monitored, and study invitations were sent to specific demographic groups, with the goal of enrolling a sample of 800 participants in the longitudinal study: equal distribution across eight, 10-year age bands (ages 18-80+); 60% with <16 years of education; 10% non-Latinx Black, 15% Latinx, and 5% non-White other ethnocultural identity.
Results:Between January-June 2022, 48,110 BHR participants were invited to the MTB study. Of those, 8294 (17%) expressed interest, 3401 (7%) completed the baseline assessment, 850 (1.8%) were assigned to the longitudinal study, and 782 (1.6%) completed a short-term longitudinal assessment. Study staff received 797 help tickets submitted by participants asking for email support to complete MTB. The baseline cohort had and average age of 64 years and an average of 16.6 years of education, 76.2% female, 2.1% non-Latinx Black, 7.1% Latinx, 86.8% non-Latinx White, and 4% from other ethnocultural groups. The longitudinal cohort had an average age of 62.3 years and an average of 16.1 years of education, 80% female, 2.8% non-Latinx Black, 8.5% Latinx, 83.5% non-Latinx White; and 5% other ethnocultural group. Compared to those invited to the study, those who enrolled in the longitudinal study were older, had higher educational attainment, and were more likely to be female and self-identify as non-Latinx White (p<0.05 for all).
Conclusions:Efficient enrollment and task completion of a large cohort in a novel, app-based mobile cognitive assessment is feasible in a completely remote setting. Most participants were able to complete MTB without individual support, indicating good usability. This approach can be scaled up to efficiently assess cognition in many research and healthcare settings. A remaining challenge is achieving robust ethnocultural and educational diversity.
A psychometric evaluation of the NIH Toolbox fluid cognition tests adapted for Swahili and Dholuo languages in Kenyan children and adolescents
- Megan S. McHenry, Anna Roose, Emily Abuonji, Mark Nyalumbe, David Ayuku, George Ayodo, Tuan M. Tran, Aaron J. Kaat
-
- Journal:
- Journal of the International Neuropsychological Society / Volume 29 / Issue 10 / December 2023
- Published online by Cambridge University Press:
- 22 November 2023, pp. 933-942
-
- Article
- Export citation
-
Objective:
Our objective was to evaluate the psychometric properties of the culturally adapted NIH Toolbox African Languages® when used in Swahili and Dholuo-speaking children in western Kenya.
Method:Swahili-speaking participants were recruited from Eldoret and Dholuo-speaking participants from Ajigo; all were <14 years of age and enrolled in primary school. Participants completed a demographics questionnaire and five fluid cognition tests of the NIH Toolbox® African Languages program, including Flanker, Dimensional Change Card Sort (DCCS), Picture Sequence Memory, Pattern Comparison, and List Sorting tests. Statistical analyses examined aspects of reliability, including internal consistency (in both languages) and test–retest reliability (in Dholuo only).
Results:Participants included 479 children (n = 239, Swahili-speaking; n = 240, Dholuo-speaking). Generally, the tests had acceptable psychometric properties for research use within Swahili- and Dholuo-speaking populations (mean age = 10.5; SD = 2.3). Issues related to shape identification and accuracy over speed limited the utility of DCCS for many participants, with approximately 25% of children unable to match based on shape. These cultural differences affected outcomes of reliability testing among the Dholuo-speaking cohort, where accuracy improved across all five tests, including speed.
Conclusions:There is preliminary evidence that the NIH Toolbox ® African Languages potentially offers a valid assessment of development and performance using tests of fluid cognition in Swahili and Dholuo among research settings. With piloting underway across other diverse settings, future research should gather additional evidence on the clinical utility and acceptability of these tests, specifically through the establishment of norming data among Kenyan regions and evaluating these psychometric properties.